为了使用各种类型的数据理解现实世界，人工智能（AI）是当今最常用的技术。在分析数据中找到模式的同时表示主要任务。这是通过提取代表性特征步骤来执行的，该步骤是使用统计算法或使用某些特定过滤器进行的。但是，从大规模数据中选择有用的功能代表了至关重要的挑战。现在，随着卷积神经网络（CNN）的发展，功能提取操作变得更加自动和更容易。 CNN允许处理大规模的数据，并涵盖特定任务的不同方案。对于计算机视觉任务，卷积网络也用于为深度学习模型的其他部分提取功能。选择合适的网络用于特征提取或DL模型的其他部分不是随机工作。因此，这种模型的实现可能与目标任务以及其计算复杂性有关。已经提出了许多网络，并成为任何AI任务中任何DL模型的著名网络。这些网络被利用用于特征提取或在任何名为骨架的DL模型的开头。骨干是以前在许多其他任务中训练并证明其有效性的已知网络。在本文中，现有骨干的概述，例如详细说明给出了VGG，Resnets，Densenet等。此外，通过对所使用的骨干进行审查，讨论了几个计算机视觉任务。此外，还基于每个任务的骨干，还提供了性能的比较。

With Twitter's growth and popularity, a huge number of views are shared by users on various topics, making this platform a valuable information source on various political, social, and economic issues. This paper investigates English tweets on the Russia-Ukraine war to analyze trends reflecting users' opinions and sentiments regarding the conflict. The tweets' positive and negative sentiments are analyzed using a BERT-based model, and the time series associated with the frequency of positive and negative tweets for various countries is calculated. Then, we propose a method based on the neighborhood average for modeling and clustering the time series of countries. The clustering results provide valuable insight into public opinion regarding this conflict. Among other things, we can mention the similar thoughts of users from the United States, Canada, the United Kingdom, and most Western European countries versus the shared views of Eastern European, Scandinavian, Asian, and South American nations toward the conflict.

Video capture is the most extensively utilized human perception source due to its intuitively understandable nature. A desired video capture often requires multiple environmental conditions such as ample ambient-light, unobstructed space, and proper camera angle. In contrast, wireless measurements are more ubiquitous and have fewer environmental constraints. In this paper, we propose CSI2Video, a novel cross-modal method that leverages only WiFi signals from commercial devices and a source of human identity information to recover fine-grained surveillance video in a real-time manner. Specifically, two tailored deep neural networks are designed to conduct cross-modal mapping and video generation tasks respectively. We make use of an auto-encoder-based structure to extract pose features from WiFi frames. Afterward, both extracted pose features and identity information are merged to generate synthetic surveillance video. Our solution generates realistic surveillance videos without any expensive wireless equipment and has ubiquitous, cheap, and real-time characteristics.

Brain-inspired computing proposes a set of algorithmic principles that hold promise for advancing artificial intelligence. They endow systems with self learning capabilities, efficient energy usage, and high storage capacity. A core concept that lies at the heart of brain computation is sequence learning and prediction. This form of computation is essential for almost all our daily tasks such as movement generation, perception, and language. Understanding how the brain performs such a computation is not only important to advance neuroscience but also to pave the way to new technological brain-inspired applications. A previously developed spiking neural network implementation of sequence prediction and recall learns complex, high-order sequences in an unsupervised manner by local, biologically inspired plasticity rules. An emerging type of hardware that holds promise for efficiently running this type of algorithm is neuromorphic hardware. It emulates the way the brain processes information and maps neurons and synapses directly into a physical substrate. Memristive devices have been identified as potential synaptic elements in neuromorphic hardware. In particular, redox-induced resistive random access memories (ReRAM) devices stand out at many aspects. They permit scalability, are energy efficient and fast, and can implement biological plasticity rules. In this work, we study the feasibility of using ReRAM devices as a replacement of the biological synapses in the sequence learning model. We implement and simulate the model including the ReRAM plasticity using the neural simulator NEST. We investigate the effect of different device properties on the performance characteristics of the sequence learning model, and demonstrate resilience with respect to different on-off ratios, conductance resolutions, device variability, and synaptic failure.

Large language models (LLMs) have been shown to be able to perform new tasks based on a few demonstrations or natural language instructions. While these capabilities have led to widespread adoption, most LLMs are developed by resource-rich organizations and are frequently kept from the public. As a step towards democratizing this powerful technology, we present BLOOM, a 176B-parameter open-access language model designed and built thanks to a collaboration of hundreds of researchers. BLOOM is a decoder-only Transformer language model that was trained on the ROOTS corpus, a dataset comprising hundreds of sources in 46 natural and 13 programming languages (59 in total). We find that BLOOM achieves competitive performance on a wide variety of benchmarks, with stronger results after undergoing multitask prompted finetuning. To facilitate future research and applications using LLMs, we publicly release our models and code under the Responsible AI License.

Image super-resolution is a common task on mobile and IoT devices, where one often needs to upscale and enhance low-resolution images and video frames. While numerous solutions have been proposed for this problem in the past, they are usually not compatible with low-power mobile NPUs having many computational and memory constraints. In this Mobile AI challenge, we address this problem and propose the participants to design an efficient quantized image super-resolution solution that can demonstrate a real-time performance on mobile NPUs. The participants were provided with the DIV2K dataset and trained INT8 models to do a high-quality 3X image upscaling. The runtime of all models was evaluated on the Synaptics VS680 Smart Home board with a dedicated edge NPU capable of accelerating quantized neural networks. All proposed solutions are fully compatible with the above NPU, demonstrating an up to 60 FPS rate when reconstructing Full HD resolution images. A detailed description of all models developed in the challenge is provided in this paper.

Self-supervised pre-training recently demonstrates success on large-scale multimodal data, and state-of-the-art contrastive learning methods often enforce the feature consistency from cross-modality inputs, such as video/audio or video/text pairs. Despite its convenience to formulate and leverage in practice, such cross-modality alignment (CMA) is only a weak and noisy supervision, since two modalities can be semantically misaligned even they are temporally aligned. For example, even in the commonly adopted instructional videos, a speaker can sometimes refer to something that is not visually present in the current frame; and the semantic misalignment would only be more unpredictable for the raw videos from the internet. We conjecture that might cause conflicts and biases among modalities, and may hence prohibit CMA from scaling up to training with larger and more heterogeneous data. This paper first verifies our conjecture by observing that, even in the latest VATT pre-training using only instructional videos, there exist strong gradient conflicts between different CMA losses within the same video, audio, text triplet, indicating them as the noisy source of supervision. We then propose to harmonize such gradients, via two techniques: (i) cross-modality gradient realignment: modifying different CMA loss gradients for each sample triplet, so that their gradient directions are more aligned; and (ii) gradient-based curriculum learning: leveraging the gradient conflict information on an indicator of sample noisiness, to develop a curriculum learning strategy to prioritize training on less noisy sample triplets. Applying those techniques to pre-training VATT on the HowTo100M dataset, we consistently improve its performance on different downstream tasks. Moreover, we are able to scale VATT pre-training to more complicated non-narrative Youtube8M dataset to further improve the state-of-the-arts.

有效的缩放和灵活的任务接口使大型语言模型能够在许多任务中表现出色。帕利（Pali）根据视觉和文本输入生成文本，并使用该界面以许多语言执行许多视觉，语言和多模式任务。为了训练帕利，我们利用了大型的编码器语言模型和视觉变压器（VITS）。这使我们能够利用其现有能力，并利用培训它们的大量成本。我们发现，视觉和语言组成部分的联合缩放很重要。由于现有的语言变压器比其视觉对应物要大得多，因此我们训练迄今为止最大的VIT（VIT-E），以量化甚至大容量视觉模型的好处。为了训练Pali，我们基于一个新的图像文本训练集，其中包含10B图像和文本，以100多种语言来创建大型的多语言组合。帕利（Pali）在多个视觉和语言任务（例如字幕，视觉问题，索方式，场景文本理解）中实现了最新的，同时保留了简单，模块化和可扩展的设计。

许多NLP任务受益于使用通常具有超过1000亿参数的大语言模型（LLM）。随着Bloom-176b和Opt-175B的发布，每个人都可以下载该规模的预估计型号。尽管如此，使用这些模型仍需要许多研究人员无法获得高端硬件。在某些情况下，LLM可以通过RAM卸载或托管API更实惠。但是，这些技术具有先天的局限性：对于交互推理而言，卸载太慢，而API的灵活性不足以进行研究。在这项工作中，我们通过加入信任处理客户数据的多个政党的资源来提出花瓣$ - $ $用于推理和微调大型模型的系统。我们证明，这种策略的表现极大地超过了非常大型型号的卸载，以每秒约1美元的价格$ \ $ \ $ \ $ \ $ \ $ \ $ \ $ \ $ 1。与大多数推理API不同，花瓣还本地揭示了服务模型的隐藏状态，从而使其用户可以根据有效的微调方法训练和共享自定义模型扩展。

在恶性原发性脑肿瘤中，癌细胞浸润到周围的脑结构中，导致不可避免的复发。对周围区域的浸润性异质性（活检或切除可能是危险的区域）的定量评估对于临床决策很重要。以前关于表征周围区域浸润性异质性的工作使用了各种成像方式，但是已经探索了细胞外无水运动限制的信息。在这里，我们通过使用基于扩散的张量成像（DTI）的自由水量分数图来表征一组独特的人工智能（AI）标记，从而捕获肿瘤浸润的异质性，从而捕获肿瘤的异质性。首先通过利用胶质母细胞瘤和脑转移的广泛不同的水扩散性能作为在周围肿瘤组织中有和没有浸润的区域的区域，首先提取了一种新型的基于体素的深度学习周围微环境指数（PMI）。均匀高PMI值的局部枢纽的描述性特征被提取为基于AI的标记，以捕获渗透性异质性的不同方面。提出的标记物应用于两个临床用例，对275个成人型弥漫性神经胶质瘤的独立人群（4级）分析，分析异氯酸盐 - 脱水酶1（IDH1） - wildtypes之间的生存持续时间以及带有IDH1-杀剂的差异。我们的发现提供了一系列标记物作为浸润的替代物，可捕获有关周围微观结构异质性生物学潜在生物学的独特见解，使其成为与生存和分子分层有关的预后生物标志物，并具有潜在的适用性在临床决策中。